{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Demo of ROCKET transform\n",
    "\n",
    "## Overview\n",
    "\n",
    "ROCKET [1] transforms time series using random convolutional kernels (random length, weights, bias, dilation, and padding). ROCKET computes two features from the resulting feature maps: the max, and the proportion of positive values (or ppv). The transformed features are used to train a linear classifier.\n",
    "\n",
    "[1] Dempster A, Petitjean F, Webb GI (2019) ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernels. [arXiv:1910.13051](https://arxiv.org/abs/1910.13051)\n",
    "\n",
    "***\n",
    "\n",
    "## Contents\n",
    "\n",
    "1. Imports\n",
    "2. Univariate Time Series\n",
    "3. Multivariate Time Series\n",
    "4. Pipeline Example\n",
    "\n",
    "***\n",
    "\n",
    "## 1 Imports\n",
    "\n",
    "Import example data, ROCKET, and a classifier (`RidgeClassifierCV` from scikit-learn), as well as NumPy and `make_pipeline` from scikit-learn.\n",
    "\n",
    "**Note**: ROCKET compiles (via Numba) on import, which may take a few seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:46.441933Z",
     "iopub.status.busy": "2020-12-19T14:32:46.441213Z",
     "iopub.status.idle": "2020-12-19T14:32:46.443225Z",
     "shell.execute_reply": "2020-12-19T14:32:46.444014Z"
    }
   },
   "outputs": [],
   "source": [
    "# !pip install --upgrade numba"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:46.448396Z",
     "iopub.status.busy": "2020-12-19T14:32:46.447602Z",
     "iopub.status.idle": "2020-12-19T14:32:51.904418Z",
     "shell.execute_reply": "2020-12-19T14:32:51.905034Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from sklearn.linear_model import RidgeClassifierCV\n",
    "from sklearn.pipeline import make_pipeline\n",
    "\n",
    "from sktime.datasets import load_arrow_head  # univariate dataset\n",
    "from sktime.datasets import load_basic_motions  # multivariate dataset\n",
    "from sktime.transformations.panel.rocket import Rocket"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 Univariate Time Series\n",
    "\n",
    "We can transform the data using ROCKET and separately fit a classifier, or we can use ROCKET together with a classifier in a pipeline (section 4, below).\n",
    "\n",
    "### 2.1 Load the Training Data\n",
    "For more details on the data set, see the [univariate time series classification notebook](https://github.com/alan-turing-institute/sktime/blob/main/examples/02_classification_univariate.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:51.908710Z",
     "iopub.status.busy": "2020-12-19T14:32:51.908101Z",
     "iopub.status.idle": "2020-12-19T14:32:51.918987Z",
     "shell.execute_reply": "2020-12-19T14:32:51.919508Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train, y_train = load_arrow_head(split=\"train\", return_X_y=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 Initialise ROCKET and Transform the Training Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:51.923023Z",
     "iopub.status.busy": "2020-12-19T14:32:51.922451Z",
     "iopub.status.idle": "2020-12-19T14:32:52.164365Z",
     "shell.execute_reply": "2020-12-19T14:32:52.164864Z"
    }
   },
   "outputs": [],
   "source": [
    "rocket = Rocket()  # by default, ROCKET uses 10,000 kernels\n",
    "rocket.fit(X_train)\n",
    "X_train_transform = rocket.transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3 Fit a Classifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We recommend using `RidgeClassifierCV` from scikit-learn for smaller datasets (fewer than approx. 20K training examples), and using logistic regression trained using stochastic gradient descent for larger datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:52.168847Z",
     "iopub.status.busy": "2020-12-19T14:32:52.168155Z",
     "iopub.status.idle": "2020-12-19T14:32:52.284816Z",
     "shell.execute_reply": "2020-12-19T14:32:52.285506Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "RidgeClassifierCV(alphas=array([1.00000000e-03, 4.64158883e-03, 2.15443469e-02, 1.00000000e-01,\n       4.64158883e-01, 2.15443469e+00, 1.00000000e+01, 4.64158883e+01,\n       2.15443469e+02, 1.00000000e+03]),\n                  normalize=True)"
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier = RidgeClassifierCV(alphas=np.logspace(-3, 3, 10), normalize=True)\n",
    "classifier.fit(X_train_transform, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.4 Load and Transform the Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:52.289448Z",
     "iopub.status.busy": "2020-12-19T14:32:52.288717Z",
     "iopub.status.idle": "2020-12-19T14:32:53.307829Z",
     "shell.execute_reply": "2020-12-19T14:32:53.308341Z"
    }
   },
   "outputs": [],
   "source": [
    "X_test, y_test = load_arrow_head(split=\"test\", return_X_y=True)\n",
    "X_test_transform = rocket.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.5 Classify the Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:53.312125Z",
     "iopub.status.busy": "2020-12-19T14:32:53.311628Z",
     "iopub.status.idle": "2020-12-19T14:32:53.409775Z",
     "shell.execute_reply": "2020-12-19T14:32:53.410342Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.8171428571428572"
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier.score(X_test_transform, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "## 3 Multivariate Time Series\n",
    "\n",
    "We can use ROCKET in exactly the same way for multivariate time series.\n",
    "\n",
    "### 3.1 Load the Training Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:53.413597Z",
     "iopub.status.busy": "2020-12-19T14:32:53.412786Z",
     "iopub.status.idle": "2020-12-19T14:32:53.775638Z",
     "shell.execute_reply": "2020-12-19T14:32:53.776690Z"
    }
   },
   "outputs": [],
   "source": [
    "X_train, y_train = load_basic_motions(split=\"train\", return_X_y=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Initialise ROCKET and Transform the Training Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:53.794896Z",
     "iopub.status.busy": "2020-12-19T14:32:53.794345Z",
     "iopub.status.idle": "2020-12-19T14:32:54.613570Z",
     "shell.execute_reply": "2020-12-19T14:32:54.614198Z"
    }
   },
   "outputs": [],
   "source": [
    "rocket = Rocket()\n",
    "rocket.fit(X_train)\n",
    "X_train_transform = rocket.transform(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3 Fit a Classifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:54.618359Z",
     "iopub.status.busy": "2020-12-19T14:32:54.617890Z",
     "iopub.status.idle": "2020-12-19T14:32:54.836560Z",
     "shell.execute_reply": "2020-12-19T14:32:54.837249Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "RidgeClassifierCV(alphas=array([1.00000000e-03, 4.64158883e-03, 2.15443469e-02, 1.00000000e-01,\n       4.64158883e-01, 2.15443469e+00, 1.00000000e+01, 4.64158883e+01,\n       2.15443469e+02, 1.00000000e+03]),\n                  normalize=True)"
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier = RidgeClassifierCV(alphas=np.logspace(-3, 3, 10), normalize=True)\n",
    "classifier.fit(X_train_transform, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.4 Load and Transform the Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:54.841004Z",
     "iopub.status.busy": "2020-12-19T14:32:54.840351Z",
     "iopub.status.idle": "2020-12-19T14:32:55.906455Z",
     "shell.execute_reply": "2020-12-19T14:32:55.907064Z"
    }
   },
   "outputs": [],
   "source": [
    "X_test, y_test = load_basic_motions(split=\"test\", return_X_y=True)\n",
    "X_test_transform = rocket.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.5 Classify the Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:55.910253Z",
     "iopub.status.busy": "2020-12-19T14:32:55.909743Z",
     "iopub.status.idle": "2020-12-19T14:32:56.008364Z",
     "shell.execute_reply": "2020-12-19T14:32:56.008931Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": "1.0"
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classifier.score(X_test_transform, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "## 4 Pipeline Example\n",
    "\n",
    "We can use ROCKET together with `RidgeClassifierCV` (or another classifier) in a pipeline.  We can then use the pipeline like a self-contained classifier, with a single call to `fit`, and without having to separately transform the data, etc.\n",
    "\n",
    "### 4.1 Initialise the Pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:56.012465Z",
     "iopub.status.busy": "2020-12-19T14:32:56.011939Z",
     "iopub.status.idle": "2020-12-19T14:32:56.013801Z",
     "shell.execute_reply": "2020-12-19T14:32:56.014399Z"
    }
   },
   "outputs": [],
   "source": [
    "rocket_pipeline = make_pipeline(\n",
    "    Rocket(), RidgeClassifierCV(alphas=np.logspace(-3, 3, 10), normalize=True)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Load and Fit the Training Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:56.017692Z",
     "iopub.status.busy": "2020-12-19T14:32:56.017166Z",
     "iopub.status.idle": "2020-12-19T14:32:56.420648Z",
     "shell.execute_reply": "2020-12-19T14:32:56.421247Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "Pipeline(steps=[('rocket', Rocket()),\n                ('ridgeclassifiercv',\n                 RidgeClassifierCV(alphas=array([1.00000000e-03, 4.64158883e-03, 2.15443469e-02, 1.00000000e-01,\n       4.64158883e-01, 2.15443469e+00, 1.00000000e+01, 4.64158883e+01,\n       2.15443469e+02, 1.00000000e+03]),\n                                   normalize=True))])"
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train, y_train = load_arrow_head(split=\"train\", return_X_y=True)\n",
    "\n",
    "# it is necessary to pass y_train to the pipeline\n",
    "# y_train is not used for the transform, but it is used by the classifier\n",
    "rocket_pipeline.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.3 Load and Classify the Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2020-12-19T14:32:56.425026Z",
     "iopub.status.busy": "2020-12-19T14:32:56.424348Z",
     "iopub.status.idle": "2020-12-19T14:32:57.602704Z",
     "shell.execute_reply": "2020-12-19T14:32:57.603291Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": "0.7942857142857143"
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test, y_test = load_arrow_head(split=\"test\", return_X_y=True)\n",
    "\n",
    "rocket_pipeline.score(X_test, y_test)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
